1. Field of the Invention
This invention relates generally to the field of data compression systems, and more specifically to an improved method and format for organizing, storing and transmitting compressed data.
2. Discussion of the Prior Art
Data compression has become increasingly vital in today""s computer systems due to the high demand for data transmission and storage capacity. In particular, main memory compression is now both feasible and desirable with the advent of parallel compression using a cooperative dictionary, as described in commonly-owned U.S. Pat. No. 5,729,228 to Franaszek et al. entitled PARALLEL COMPRESSION AND DECOMPRESSION USING A COOPERATIVE DICTIONARY, incorporated herein by reference. Parallel compression is a relatively new art in the field of compression. Its main concept is to divide a block of uncompressed data into multiple sectors and then assign them to individual engines for both compression and decompression with all engines sharing a cooperative dictionary such that the compression ratio is close to that of a single-engine design. This results in much better latency and throughput than the previous single-engine designs, thus making main memory compression feasible. It is the case however, that latency and throughput objectives may be better achieved provided there is implemented a highly-efficient compressed data format.
It would thus be highly desirable to provide a system and method for organizing compressed data efficiently, particularly, compressed data in parallel format, in order to enhance compression throughput and reduce decompression latency in data storage and data transmission systems.
It is an object of the invention to provide a system and method for organizing, storing and/or transmitting compressed data efficiently, particularly, by processing compressed data in parallel in order to enhance compression throughput and reduce decompression latency.
It is another object of the invention to provide a system and method that for organizing, storing and/or transmitting compressed data efficiently, particularly, by enabling a compressor mechanism to write out parallel streams of compressed data from multiple engines quickly and with minimal loss of compression ratio and, enabling a decompressor""s engines to extract the individual streams of compressed data easily without separate data delimiters and control.
Thus, according to the principles of the invention, there is provided a system and methodology for generating compressed data comprising: inputting information units to one or more parallel executing compression engines and compressing said information units into one or more compressed information units; providing a temporary storage queue associated with each compression engine for temporarily storing one or more compressed information units from its respective compression engine; collecting a compressed information unit in parallel from each temporary storage queue and merging each collected compressed information unit to form a merged word; and, successively forming merged words each comprising collected sets of compressed information units from each temporary storage queue to form a series of merged words for transmission or storage thereof.